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Abstract 

In the 60's Shapley provided an example of a two player fictitious game with periodic 
behaviour. In this game, player A aims to copy B's behaviour and player B aims to play one 
ahead of player A. In this paper wc continue to study a family of games which generalize 
Shapley's example by introducing an external parameter, and prove that there exists an 
abundance of periodic and chaotic behavior with players dithering between diflferent strate- 
gies. The reason for all this, is that there exists a periodic orbit (consisting of playing mixed 
strategies) which is of 'jttter type': such an orbit is neither attracting, repelling or of saddle 
type as nearby orbits jitter closer and further away from it in a manner which is reminiscent 
of a random walk motion. We prove that this behaviour holds for an open set of games. 
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1 Introduction 

The purpose of this paper is to show how complicated the dynamics of fictitious play can be (for 
an interpretation of fictitious play as a model for rational learning, see for example Fudenberg and 
Levine [1998]). We do this by analysing in detail the following family of 3 x 3 games determined 
by the matrices 

/lO/3\ f -(3 1 \ 

(1.1) 

which depend on a parameter P e (0, 1) (and best response dynamics given by the differential 
inclusion (1.2)). In fact, we shall show in Theorem 1.4 that our results even hold for matrices 
A, B with 

\\A ~ Ai3\\, \\B - Bp\\ < e with e > small. 

However, except for Theorem 1.4 and Section 6, we shall simply write A = Afj and B = Bfj. 
As usual, player A has utility jAAp^ whereas player B has utility p"^Bp^ where the row vector 
G C M.^ denoted the position of player A and the column vector p-^ £ 'Eb C M.^ the 
position of player B. For later use we write = Ap^ and = p^B. Here S^, Ss are the set 
of probability vectors in M.^. In other words, S := x Eg is the product of two-dimensional 
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triangles and so topologically it is a ball in M^. Player B (resp. A) is indifferent between all three 
strategies when = E^, = and E = {E^, E^) is the Nash equilibrium of the game. For 
the game A = A^, B = Bp one has E^ := (1/3, 1/3, 1/3) and E^ := (1/3, 1/3, 1/3)'. 

For /? = the game A, B is equivalent to the classical example introduced by Shapley [1964] 
(where each of the players eventually chooses strategies periodically). For /3 = (v^— 1)/2 fa 0.618, 
the game is equivalent to a zero-sume game (rescaling B to B = a{B — 1) gives A + B = 0), so 
then [1951] play always converges to the interior equilibrium E'^^E^. 

The best response BRa{p^) of player A is the i-th unit vector if the i-th. component of 
is larger than the other components of (if several components of are equally large, then 
BRa{p^) is the convex combination of unit vectors corresponding to the largest components of 
v^). Define BRb{p'^) similarly. Once best responses are selected, the dynamics is determined 
by moving in a straight line towards the best responses. In some of the literature this is done by 
taking the piecewise linear differential equation 



dp^/dt =BRa{p^)-p^ 
dp^/dt =BRb{p^)-P 



(1.2) 



whereas others take 

dp^/ds = {1/s){BRa{p'')-p^) 

dp'^/ds = {1/s){BRb{p^)-p'^). ^ ' 

The orbits are the same in both cases, only the time parametrisation of the orbits differs (take 
s = e*); for the latter, orbits slow down and a periodic orbit of period T (1.2) corresponds to 

an orbit of (1.3) which returns in time 6^,6^-^,6^^, Equations (1.2) and (1.3) determine the 

dynamics up until such time as one or other (or both) players become indifferent between two (or 
more) pure strategies. When one or more of the players is indifferent between two strategies their 
dynamics may not be uniquely determined. So (1.2) and (1.3) are in fact differential inclusions 
rather than differential equations, but as the best response correspondences p^ BRa{p^) and 
p^ BRsip"^) are upper semicontinous with values closed, convex sets, it follows from Aubin 
and Cellina [1984, Chapter 2.1] that through each initial value there exists at least one solution 
which is Lipschitz continuous and defined for all positive time. It is shown in Hofbauer [1995] 
that, under mild regularity conditions (which are satisfied in our case), any solution is piecewise 
linear. 

In fact, when the matrices (1.1) are chosen, play is not affected at all by this ambiguity except 
at E (because a certain transversality condition is satisfied, see Sparrow [2008]). In other words, 
all the orbits (except the one through E) are uniquely determined (outside E, the dynamics is 
not affected by a choice of tie- breaking rule). Moreover, when (3 G (0,1) the flow is continuous 
except at E (when /3 ^ a, for a proof, see Sparrow et al [2008]). 

Note that the best response of A to any p^ ^ E^ is either an integer i e {1, 2, 3} or a mixed 
strategy set i where i := {1, 2, 3} \ {i} corresponding to where player A is indifferent between two 
strategies but will not play i. Similarly for B. Hence one can associate to any orbit {p"^ {t) , p^ (t) 
outside E, a sequence of times to := < ti < t2 < ■ ■ ■ and a sequence of best-response strategies 
(iojo), ihji), {12,32), ■ ■ ■ where 

{in,jn) = {BR''{p''{t),BR''{p^{t)) for t e (i„,i„+i) 

with in and j„ equal to 1, 2, 3, 1, 2 or 3 for each n = 0, 1, 2, In Sparrow et al [2008] we showed 

that there are three periodic orbits: one for /3 e (0, a) with play (1, 2), (2, 2), (2, 3), (3, 3), (3, 1), 
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(1,1) (the Shaplcy orbit), one for /3 G ((7,1) with cydic play (1,3), (1,2), (3, 2), (3, 1), (2,1), (2,3) 
and a third one with a period 6 orbit of mixed strategies (1, 1), (1, 2), (2, 2), (2, 3), (3, 3), (3, 1). 
The latter sequence of strategies correspond to a fully-invariant set C(r) (so an orbit starting in 
this set remains in this set, and an orbit starting outside this set remains outside this set); this 
fully invariant set exists for each /3 G (0, 1) and contains a periodic orbit when /3 € {a, 1). The 
latter orbit is of 'jitter type ': it is neither attracting, repelling or of saddle type; instead nearby 
orbits jitter closer and further away from it in a manner which is reminiscent of a random walk 
motion. We describe this behavior in the final sections of this paper. 

1.1 Abundance of Periodic Play 

Let us state now the main results of this paper. To do this, let us say that an orbit of the game 
has cyclic play of period n if the associated sequence (in, in) is periodic: (ik+n, jk+n) — {ik,ik) 
for all /c = 0, 1, . . . . Given k G {0, . . . ,n — 1} we say that the players are indecisive at the k-th 
step if A; > 3 and moreover 

{ik-3,ik-'2,jk-i,ik} {1,2,3} and {jk-3, jk-2, jk-i, jk} {1,2,3} 

holds (so during this and the previous three moves, both players never deviated from a choice of 
two strategies). Sometimes we also will say that the players dither at the fc-step. In the opposite 
case, we say the fc-th step is decisive. The essential period of a cyclic play of period n is the 
number of decisive steps k < n. For example, the cycle of period 6 

(1,2),(2,2),(2,3),(3,3),(3,1),(1,1) 

never dithers whereas for the cycle of period 7 

(1,2),(2,2),(2,3),(3,3),(3,2),(3,1),(1,1) 

players dither in the 5th step, so the essential period is again 6. Let a = (\/5 — l)/2 rs 0.618. 

Theorem 1.1. [An abundance of periodic play] For each (3 G (0,1) and each n > 1 there are 

infinitely many different orbits "fg, s = 1,2,... of the differential equation (1.2) (and of (1-3)) 
with corresponding cyclic play of period Ns^ooass^oo but with essential period equal to 6n. 
Moreover, 

• for 13 G (0, a), these orbits with cyclic play reach the interior equilibrium E in finite time; 

• for (5 G (cr, 1) these orbits with cyclic play are genuine periodic orbits of (1.2) (and of (1.3)). 

In Sparrow et al [2008] we showed that for (3 G (— l,cr) there exists a periodic orbit corre- 
sponding to cyclic play (1, 2), (2, 2), (2, 3), (3, 3), (3, 1), (1, 1) (the Shapley orbit) which attracts 
an open set of initial conditions; the above theorem shows that many periodic orbits are not 
attracted to this cycle. In that paper it was also shown that for (3 G (cr, 1) there exists another 
periodic orbit corresponding to cyclic play (1, 3), (1, 2), (3, 2), (3, 1), (2, 1), (2, 3) (the anti-Shapley 
orbit) which becomes attracting when ct G (t, 1) where r ~ 0.915. Again this attracting orbit 
does not attract everything. 

That the players can have infinitely many orbits with the same essential period, is a conse- 
quence of the fact that there is a sequence of periodic orbits converging to the orbit of mixed 
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strategies (1, 1), (1, 2), (2, 2), (2, 3), (3, 3), (3, 1), and which all have the same essential period. For 
these periodic orbits, the essential period is the number of times it follows the period 6 orbit 
before returning to its original position, whereas the actual period increases if the players dither 
for longer along each of the 6 legs. Along these periodic orbits at any given moment only one of 
the players is indifferent, but they dither for a long time between each decisive move. 

Let us relate this theorem to a result of Krishna and Sjostrom [1998] (which builds on earlier 
work of Rosenmiiller [1971]). In this interesting paper, they show that for a generic game (i.e. 
for Lebesgue almost all pay-off matrices) there exists no open set of initial conditions for which 
fictitious play converges cyclically to a mixed strategy equilibrium (unless both players use at most 
two pure strategies). In other words, if fictitious play converges to a mixed strategy equilibrium 
with both players using more than two strategies then the choice of strategies cannot follow 
a cyclic pattern unless possibly the initial conditions are in some codimension-one space. Our 
result shows that for /? G (0, a) countably many orbits do indeed converge cyclically to the 
equilibrium (and along these orbits at any given moment only one of the players is indifferent). 
Our result does not rely on the symmetry of the matrices: it holds for an open set of matrices, 
see Theorem 1.4 and Section 6. 

For their result, Krishna and Sjostrom only only needed to consider orbits for which at any 
given moment only one of the players is indifferent. It is interesting to note however that, as 
becomes clear from this paper, it is precisely near the set C(T) where both players are simulta- 
neously indifferent that much of the interesting behaviour happens (and this set 'organises' the 
local dynamics). 



1.2 Abundance of Dithering Behavior 

The next theorem shows that there are many orbits which dither for very long periods. To make 
this precise, let us assume the players start at p G E and aim for (zo,.7o), (*i, ji), («2, j2), • • • • 
Next associate to these moves a sequence {B.k)k>3 with Rk e {D.I}, where Bk is equal to D or 
/ depending on whether the players are decisive or indecisive at time A: > 3. In this way we get 
a map 

i:3p^{Rk{p)}k>3e{D,if 

which captures partly what play evolves from starting position p G S. (We ignore A; < 3 because 
by definition the players are then always decisive. More precisely, if T denotes the map which 
assigns to {ik,jk)k>o the sequence {Rk)k>ka and a the shift map, then T o a = a oT only if we 
take fco > 3.) 

To simplify the coding even further, define the times 3 < No{p) < Ni{p) < N2{p) < ... 
for which the players are decisive (only considering times > 3). Note that these times uniquely 
determine again the sequence i?3(p). i?4(p), ... .If Ns+i{p) — Ns{p) is large, then we say that 
the players dither for a long time (as they then each play back and forth between two strategies). 

Theorem 1.2. [There is a lot of freedom in the choice of dithering sequences] For each (3 G (0, 1) 
there exist N° gN, 0<A<1</U and a compact set X C S, so that for each sequence 
No < N2 < Ni < ... with No>3 and with 

A < "^^ ~ ^^r^"^ < IJ- and N2S+2 - N2s > N° for all s > 

N2S+2 — N2s 
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there exists p £ X such that 



\N2s{p)-N2s\ <ifor alls>0. 

Moreover, 

• for /3 e (0,(t) orbits in X converge to E; 

• for /3 e (cr, 1) orbits starting in X do not converge to E. 

That this theorem only refers to the gaps between even decisive moments, N2S+2 — -^2s, is 
because the gaps between the even and odd moments are somewhat more arbitrary. However, if 
N2s+2{p) — N2s{p) is large for all .s > 0, then Ns+i{p) — Ns{p) is also large for all s > 0, i.e., the 
players dither for long periods between making a decisive move for orbits described above. 

We did some numerical simulations for games determined by completely different matrices. 
In many of these, similar dithering behaviour also occurred. Even for zero-sum games, the 
players seem to converge to equilibria in a dithering fashion (in fact, in 2 x n games, dithering is 
unavoidable) . We will report on these simulations in a subsequent paper. 

1.3 Chaotic Behavior 

The previous theorem states that there are orbits starting in X which dither for more or less 
arbitrary lengths A'2,,+2 ^ Ng. An immediate application of this theorem is the following result: 

Theorem 1.3 (Chaos). Take N > N° so large that X < {N -1)/N < {N +l)/N < jj. For each 
sequence {e2s)s>o with e2s S {—5,0, 5} there exists p G X with 

N2S+2 - N2s e[N + e2s, ^ + 4 + 62.] for all s > 0. 

So for such a p, N2S+2 - N2S is in the interval [N -5,N - 1], [N, TV + 4] or [N + 5,N + 9] 
depending on the parity of e2s ■ In particular the flow contains subshifts of finite type and has 
positive topological entropy. The flow also has sensitive dependence on initial conditions. 

The definition of the notions 'subshifts of finite type', 'positive topological entropy' and 'sen- 
sitive dependence on initial conditions' can be found in almost any book on dynamical systems, 
for example Guckenheimer and Holmes [1983]. We have numerical evidence that this game is 
chaotic in a more profound sense: it appears that there exists a range of parameters /3 G (cr, t) 
so that (Lebesgue) almost all starting positions correspond to chaotic behaviour. We will report 
on this in a subsequent paper. 

1.4 Robustness 

The above results do not require the matrices to be of a special form, and hold for games 

corresponding to an open set of matrices: 

Theorem 1.4 (Robustness). For each (3 G (0, 1) with f3 ^ a, there exists e > so that for each 
3x3 matrices A and B with 

\\A-A0MB-B0\\<e 
the previous theorems also hold. More precisely, 
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• for /3 e (0,(t) and e > sufficiently small, 

— there exists a periodic orbit corresponding to cyclic play (1,2), (2,2), (2,3), (3,3), 
(3,1), (1,1) (the Shapley orbit) which attracts an open set of initial conditions; 

— there exist orbits of mixed strategies (1, 1), (1, 2), (2, 2), (2, 3), (3, 3), (3, 1); such orbits 
lie on a cone with apex E; all orbits on this cone converge to E; there are infinitely 
many different orbits -fs, s = 1,2,... as in Theorem 1.1 which reach the interior 

equilibrium E in finite time; 

— there are orbits which dither as in Theorem 1.2 which again reach the interior equilib- 
rium E in finite time; 

• for /3 e (a, 1) and e > sufficiently small, 

— there exist a periodic orbit T of mixed strategies (1, 1), (1, 2), (2, 2), (2, 3), (3, 3), (3, 1); 
the cone through this periodic orbit with apex E is completely invariant and all orbits 
on this cone ( apart from, E ) converge to this periodic orbit; 

— there are infinitely many different periodic orbits 7s, s = 1, 2, . . . as in Theorem 1.1 
(these orbits stay nearV); 

— there are orbits which dither as in Theorem 1.2 (these orbits also stay nearV); 

— one has chaos and sensitive dependence on initial conditions; 

• for P G (t, 1) and e > sufficiently small, there exists an attracting orbit which correspond- 
ing to cyclic play (1, 3), (1, 2), (3, 2), (3, 1), (2, 1), (2,3) (the anti-Shapley orbit). 

Here r « 0.915 is the root of some polynomial of degree 6, which we computed in Spaxrow et 
al [2008]. Of course A and B near A/} resp. -B/3 will have a Nash equilibrium E, which is close 
but not necessarily equal to the Nash equilibrium of Ap, Bp. For matrices A.B near Acr,B„ 
the existence of periodic orbits and of a dithering set also hold, but it is no longer clear whether 
these orbits converge to the Nash equilibria E or not. 

1.5 The idea of the proof and some general comments 

The main point of our analysis is to exploit that one can simplify the study by identifying 
points on half-lines through E. This way we get an induced flow on (which is topologi- 
cally a three sphere). Associated to each set Y C 9S which is forward invariant under the 
induced flow is the cone C{Y) over Y with apex E which is forward invariant under the original 
flow. Similarly, a periodic orbit 7 of the original flow, corresponds to a periodic orbit 7 of the 
induced flow. We apply this idea in particular to the periodic orbit T with mixed strategies 
(1, 1), (1, 2), (2, 2), (2, 3), (3, 3), (3, 1) and the corresponding periodic orbit F of the induced flow. 
It turns out that a flrst return map to a section through a point in F has extremely interesting 
behaviour: it is of 'jitter type', see the flnal section of this paper. 

We believe that looking at our approach of analysing the induced flow, and the notion of 
orbits of 'jitter type' will be useful in analysing flctitious play in general. 

This is not the first time subshifts of finite type were shown to exist in fictitious play. Cowan 
[1992] already did this, by considering a matrix with extremely large coeflicients. Our work is 
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closer in spirit to Bcrgcr [1995] who considers a family of symmetric bimatrix games depending 
on a parameter k such that (i) for k G (—2,0) has a Shapley orbit which degenerates as A: ^ 0, 
and (ii) for fc > 0, as in our case, there exists a hexagonal orbit along which both players are 
indifferent between (at least) 2 strategies. Berger observes similar 'chaotic' numerical phenomena 
as we did in our previous paper Sparrow et al [2008] and also shows the existence of an additional 
periodic orbit of saddle-type. 

As mentioned, we have numerical evidence that there exists a range of parameters (3 G (cr, t) 
so that (Lebesgue) almost all starting positions correspond to chaotic behaviour. More general 
games also show up the same dithering behaviour. 

There are many papers which show that one has convergence to the equilibrium for games 
where one or both of the players have only 2 strategies to choose from, see Miyasawa [1961] and 
Metrick & Polak [1994] for the 2 x 2 case; Sela [2000] for the 2 x 3 case; and Berger [2005] for 
the general 2 x n case. Jordan [1993] constructed a 2 x 2 x 2 fictitious game with a stable limit 
cycle. The 3x3 example studied in this paper, shows that the situation is far more complicated 
in general. 

2 Basic results on the Shapley system 

Let us recall some results from Sparrow et all [2008]. Let us denote the set where player A is 

indifferent between strategies P/^ and by Zf^ C and define Z^^ similarly. Figure 1 shows 
pictures of the phase space marking the lines Zf^ and (for /3 > 0). Note that for all values 
of (3 both players are indifferent between all three strategies at the point E = {E^,E^) where 
E^ = {E^)^ = {1/3,1/3,1/3). 

As mentioned, for /3 e [0, a) the game has a periodic orbit with cyclic play (1,2), (2,2), (2, 3), 
(3,3), (3,1), (1,1) and this orbit attracts an open set. For f3 G (c, 1) the game has another 
periodic orbit with cyclic play (1, 3), (1, 2), (3, 2), (3, 1), (2, 1), (2, 3) and this orbit is attracting 
for P G (r, 1). When (3 a these orbits shrink to E. A third periodic orbit F exists when 
13 G (cr, 1) with periodic 6 cyclic play (1,1), (1,2), (2,2), (2,3), (3,3), (3,1). For 13 G (0,ct) there 
still exist orbits with this periodic 6 play, but these orbit are not periodic, instead they converge 
to E. Two of the six sides of the corresponding hexagon are schematically drawn in Figure 1. 
Note that F is contained in 

J := {Z^^ X Z^^) U {Zl^ X Zt^^) \J{Zl, X Z^^) U {Z^^ x Z^,) |J 

\J{Zl,xZ^^,)U{Zl,xZi^,) (2.4) 

We call this the Jitter set, as nearby orbits jitter back and forth between strategies. When /? — > cr 
these orbits all shrink to E. 

3 The induced dynamics on 5S for the Shapley family 

In order to get a better understanding of the geometric and topological structure of all orbits in 
the Shapley family we will now consider an induced flow on i9S. This is obtained by projecting 
the original flow on S \ {E} onto dT, using the projection tt : S \ {E} dT, obtained by defining 
TT (p) to be the unique point of dT, that lies on the half line through E in the direction p. Since 
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Figure 1; The simplices and Ss and the periodic orbit on the jitter set J (for /3 e {a, 1)). Two of 
the six legs of the periodic orbit are drawn in the figure (contained in Z12 x -^i^a and Z12 x ^^^2); after 
this piece of the orbit, the orbit turns in Ea clockwise to the leg containing f^f and in Es it doubles 
back onto itself. The points f^, f^i give the maximum extent of the periodic orbit on J and Rfj and 
Rfj are the intersections of J with the boundary of Sa resp. Es. In this way the periodic orbit forms a 
hexagon in the four-dimensional space Ea x Eb which moves (cyclically) around the different legs of J. 

the best response of the two players for all points on this half-line is the same, this gives a well- 
defined flow on 9E. The new flow obtained in this way is a faithful representation of the 'angular' 
component of the original flow, but it contains no information about the radial component of the 
flow. It is easier to visualize because it is three dimensional. For example, the topological three 
sphere is homeomorphic (via stereographic projection) to the one-point compactification of 

The geometry of this induced flow will give us more insight in the original flow. Indeed let 
7 C OS and let (7(7) C E be union of the closed half- lines through E in the direction of p e 7 
(for any such p). This set wc will call the cone of j over E (this set is equal to the closure of 
7r~^(7)). Note that if 7 is a periodic orbit of the induced flow, then the cone C(7) is an invariant 
set under the original flow. 

3.1 Simple periodic orbits on the induced flow on dTi 

Even though the flow on x does not have a periodic orbit on the jitter-set J for /3 € (0, cr), 
the induced flow does have a periodic orbit on Jn 9S. The reason for this is that J is invariant, 
so the closed curve J fl 95] is an invariant set for the induced flow. When (i e (0,cr) then 
orbits in J of the original flow spiral towards {E^ , E^) (inside the topological surface J; but the 
motion is never straight towards {E^, E^)). Therefore in the induced flow this spiralling motion 
corresponds to a periodic motion on the closed curve 9S fl J. Let us denote this periodic orbit 
by f. 

We can summarize the previous results on the existence and stability of periodic orbits as 
follows. 

Proposition 3.1 (Periodic orbits for the induced flow on dYi). For (3 G (0, 1] there are (at least) 
three periodic orbits for the induced flow: the one corresponding to the clockwise periodic orbit 
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(Shapley 's orbit), an anticlockwise periodic orbit and a periodic orbit T corresponding to the Jitter 

set J . Moreover, 

• for (3 € (0, cr) the clockwise periodic orbit is attracting, the anticlockwise orbit is of saddle- 
type, and the orbit T is of jitter type; 

• for /3 € (cr, r) , the clockwise periodic orbit is attracting, the anticlockwise orbit is of saddle- 
type whereas the orbit T is still of jitter type; 

• for (i G (r, 1], the clockwise periodic orbit is of saddle-type, the anticlockwise orbit is of 
attracting and the orbit T is of jitter type. 

That r is a periodic orbit of 'jitter type' means that it has the random walk behavior described 
in Theorem . 

The induced flow is not smooth near the periodic orbit F corresponding to the Jitter set J. 
Analyzing the local dynamics near F is the main purpose of this paper and is done in the final 
section of this paper. 

Proof. The existence of the periodic orbits for the induced flow follows immediately from the 
existence of the corresponding orbits for the original system (which were established in Sparrow 
et al [2008]). In fact, the proof in the appendix in Sparrow ct al [2008] shows that the induced 
flow has a Shapley periodic orbit for /3 e [a, 1] even though the original flow does not (so for the 
original flow the Shapley orbit spirals to E when /3 e [cr, 1]). Similarly the other two orbits exist 
for all (3 e (0,1]. 

So let us discuss the stability type. If a periodic orbit of the original flow is attracting (or 
repelling) then obviously the corresponding periodic orbit of the induced flow is also attracting 
(repelling). If a periodic orbit 7 is originally of saddle- type then it depends on the eigendirections: 
the direction corresponding to the (invariant) cone C{^) consisting of all rays from the midpoint 
to the points on the periodic orbit 7 disappears in the induced flow. So we only need to consider 
the anticlockwise periodic orbit which we will still denote by 7. In the appendix of Sparrow et al 
[2008] it was shown that 7 consists of three line segments in S and we computed the linear part of 
the Poincare transition map of 7 with sections taken in indifference planes. These sections were 
taken at two symmetrically positioned distinct points computing in this way only the transition 
along one third of 7. Because of the symmetry of the system, the actual return map to a section 
Z is the third iterate of the linear map computed in that appendix. Note that Z was chosen 
to be contained in one of the indifference sets and so Z contains E. It was shown in Sparrow 
[2008] that two eigenvalues are negative, and one positive (which was equal to n2/ni). The 
positive eigenvalue remains in (0, 1) for all (3 G (cr, 1), one of the negative eigenvalues remains in 
(—1,0) for all /? G (cr, 1) whereas the other negative eigenvalue is in (—00, —1) for E (cr, t) and 
in (—1,0) for (3 e (t, 1)- For the induced flow, the flrst return map has only two eigenvalues. 
Let us explain why the positive eigenvalue corresponds to an eigenvector which lies in the cone 
(7(7) and which therefore disappears after projecting to 9E. Indeed, one of the eigenvahics of 
the linearisation at z := Z n 7 of the Poincare map P: Z ^ Z lies in the cone C(7) (because 
this cone is invariant under the flow), and so this eigenvalue is along the hue segment C{'y)r\ Z 
through z. This cone over the triangle 7 consists of a surface made up of three (two-dimensional) 
triangles in S, and so the corresponding eigenvalue is positive (since the flow preserves C{j) and 
therefore an orbit starting on one side of 7 C C(7) remains on that side). It follows that in the 
induced flow the positive eigenvalue disappears under the projection. The other two eigenvectors 
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arc projected, and their eigenvalues remain exactly the same under the projection. The original 
eigenvalue equation now is solved modulo the direction corresponding the projection, and so the 
two other eigenvalues stay the same for the induced flow. It follows that 7 is a saddle orbit if 
and only if the corresponding orbit for the induced flow is a saddle orbit. □ 

3.2 Many additional periodic orbits and chaos for the induced flow 

The next theorem is about the first return map to a section Z based at some point in F. It shows 
that orbits under this flrst return map can move closer and further away from the flxed point. 
More precisely, orbits can jump between the annuli {z € Z;dist(z,zo) G (^jt^j St)} f^round zq 
in rather free way. Here ki corresponds to the number from Theorem 1.2. We refer to this 
behavior as of 'jitter type'. 

Theorem 3.1. Take (i G (0, 1) and consider the "periodic orbit T for the induced flow correspond- 
ing to the jitter set J. Let Z he a two-dimensional surface in dYi through some point zq & T which 
is transversal to the induced flow, and let P be the Poincare first return map of the induced flow 
to Z. Then 

• for each period n e N, there are infinitely many periodic orbits of P of period n ( and one 
can even choose a sequence of such periodic orbits so that the distance of the whole orbit to 
zo converges to zero); 

• there exist orbits which jitter in the following sense: there exist a sum metric dist in Z and 
< X < 1 < H and Nq so that for each sequence ki so that 

k+i 

X < — < /X and ki > Nq, 

Ki 

there exists z G Z\r with 

dist{P\z),Zo) e (^^, ^) for all i > 0. 

• the return map P has subshifts of finite type, positive topological entropy and has sensitive 

dependence on initial conditions. 

Proof. We will give the proof of this theorem in Section 5. □ 

Denote the period of the periodic orbit F under the induced flow by T. Since the induced 
flow is continuous, a periodic orbit r„ which corresponds to a period n of the flrst return map P 
and which is close to F has, under the induced flow, period approximately n ■ T (and the closer 
the orbit is chosen to F the better this approximation is). Hence 

Corollary 3.1. Take /3 e (0, 1) and consider the periodic orbit F for the induced flow corre- 
sponding to the jitter set J. Let n gN be arbitrary. There exists a sequence of periodic orbits Tk, 
k = 1,2, . . . for the induced flow arbitrarily close to F whose period converges to nT as fc —> 00. 

Furthermore, 

Corollary 3.2. Take (3 € (0,1). The induced flow contains subshifts of finite type, positive 
topological entropy and has sensitive dependence on initial conditions. 
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3.3 Global return section 



Figure 2: The subset of (9S where players A and B are indifferent, where we identify dT, with 
U {00} . Part of the periodic orbit T is drawn as a fat curve. Another view of this periodic 
orbit is in Figure 3 (rotated 180 degrees about the z-axis). 

Remember that S is a ball in M"' and dH is homeoniorphic to . This set can be thought of as 
M^U{oo}. Usually it is not easy to easy to give a geometric image of dynamics on S^. But in fact, 
we are lucky. Instead of taking a section through a point z £ T transversal to the periodic orbit F, 
we can find a set S which is topologically a disc, such that dS — F, and such that orbits cross S\T 
transversally. This disc lies in the indifference sets. The subset of dT. where player A is indifferent 
between strategies i and j is equal to (SS/i x Z;^) U (S^ x (Z;^ n This two-dimensional 

set is equal to a triangular tube together with a triangle at one end of the tube, in other words, 
it is homeomorphic to a topological disc. The boundary of this disc corresponds to the triangle 
O'Sa X . Note that for each pair i,j the boundary of this disc is the same. So if we identify 

with M"^ U {00}, the set where player A is indifferent between two strategies can be thought of 
as the union of the upper and lower part of the unit sphere in M.^ (i.e. {{x, y, z) ; + + z"^ = 
1 and z 7^ 0}) and the disc {(x,?/, z) ; + < 1 and z — 0}. Similarly, the set where B is 
indifferent can then be thought of as {{x,y,z) ; {x,y) — (r cos 0, r sin 0) with r > and (j) — 
27r/3, 47r/3, 0} U {00}. The choice for (j) represents which of the two strategies are indifferent for 
B. Again this represents three discs which all meet along the circle {{x,y, z) ; a; = y = 0}U{oo} 
in K'^ U {00}. The orbit F lies on the intersection of the sets where A and B are indifferent and 
is drawn in Figure 3. It turns out that one can find a subset S of the space where player A is 
indifferent, such that dS = F and for which the orbits go through S transversally. 
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Figure 3: The periodic orbit f in the Jitter set drawn in 9E where we identify dT, with M'^Ujoo}. 
The vertical line is not part of the curve, but is drawn for clarity and because it forms the part 
where the four pieces from the global return section, introduced in Proposition 3.2 meet (the 
half-disc, and the two quarter discs, which meet along the vertical axis). The global return 
surface S is the surface consisting of the half-disc, and two quarter discs meeting at the z-axis. 
The return section Z mentioned in Theorem 3.1 is transversal to F and so is not contained in S. 



Proposition 3.2. There exists a topological disc S in dT, with the following properties 
m S is a piecewise linear; 

• dS^T; 

• each orbit of the induced flow ( except F ) intersects F transver sally; 

• the Poincare return map to S is a well-defined homeomorphism. 

Proof. Let S consists of four pieces within the linear indifference sets x Zf^ . These pieces are 
X Zj*3, U'i X Z^3, Ui X Zi 2j Ui X Z^i, where Ui is the part of where player B prefers to 
head for corner Pf'. The flow is transversal to each of these four linear sets. Also, inspection in 
the diagram of Figure 7 in Sparrow et al [2008] shows that no orbit can miss these sets (except 
if it is in J) . The section S forms a disc made up from the semi-disc and the two-half semi-discs 
depicted in Figure 3. □ 

Instead as in Figure 3, we can also visualise the section S as in Figure 4. Indeed, S C 9E is 
made up of the following regions: 

1. (the two triangles in E^ where player B heads for 1 resp. 3) x (the point i?f2); 

2. (the part of O'Sa where player B heads for 1 resp. 3) x (the segment [Rf2,E-^] in Eg); 



12 



3. (the part of (JSa where player B heads for 3) x (the segment [Rfj,E^] in E^); 

4. (the triangle where player B heads for 3) x (the point Rf^)', 

5. (the part of 9Syi where player B heads for 1) x (the segment [R^i,E^] in E^); 

6. (the part of where player B heads for 1) x (the point i?^3). 

Using polar-like coordinates based at E"^ e S^, the region 1 can be represented as an isosceles 
triangle (with, say, angles 80,50,50). Attaching region 2 then gives a similar triangle (which is 
the bigger triangle in Figure 4) . Joining all these regions together appropriately gives that S can 
also be represented as in Figure 4. In this figure we also show an orbit of the first return map 
for (3 = a (the zero-sum case). The orbits, which tend to the Nash equilibrium in the full flow, 
do so in a rather chaotic fashion. 




-2 -1.5 -1 -0.5 0.5 1 1.5 2 



Figure 4: The orbit of a typical starting point under the first return map to S when [3 = a. The 
dynamics is similar to that of an area preserving map. There are two 'egg-shaped' regions which 
are permuted by the first return map. Orbits within this region form invariant circles. 



4 Dynamics for the original flow in S 

Let us now state the implications of the results on the induced flow from the last section for the 
original flow. 

4.1 Invariant cones 

The implications of Theorem 3.1 for the real flow in E^ x E^ depend on the value of /3, as 
described in the following proposition. 

Proposition 4.1. For the original flow the statements from Theorem 3.1 still hold in the fol- 
lowing sense. 
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• For (3 € (0, cr) each periodic orbit 7 near T of the induced flow, corresponds to an invariant 
cone C(7). Each orbit of the original flow starting in such a cone reaches the interior 
equilibrium in finite time (and then remains there). 

• For (3 G {a, 1) periodic orbits near T in d'E correspond to periodic orbits near T on J. Thus 

there are infinitely many orbits of the flow in this parameter range. Moreover, there are 
orbits which jitter in the sense of the 2nd assertion of Theorem 3.1 and the flow contains 
a suhshift of finite type, has positive topological entropy and has sensitive dependence on 
initial conditions. 

Proof. In Prop A2 in Sparrow et al [2008] it was shown that each orbit in J tends to E when 
/3 e (0, cr). In fact, on page 290 of that paper it was shown that if we take a section in J, then 
orbits converge exponentially fast to zero under iterates of the Poincaxe map; moreover the time 
to spiral into the equilibrium E is finite. 

Now let y be a plane in S through a point Xq transversal to P and R be the first return 
map to V corresponding to the induced flow. Identify S with 9S x [0, 1] where (x, 0) G 9S x [0, 1] 
corresponds to E and letV = V x [0, 1]. Let R be the first return map to V of the original flow. 
Then R is of the form R{x, t) = {R{x), gx{t)). As we noted above, it was shown in Sparrow et al 
[2008] that 1 1— > Qxit) has derivative less than one when x = xq and P e (0,cr). Note that 

R^{x,t) = {R^{x),e^„.r^^^o...og,{t)) 

Since for (3 G (0,f7), is a hyperbolic attracting fixed point of • ~^ if R^{x) = 
X G &Tj and . . . , HP" ^{pc) are all sufficiently close to xo then t \ — ^ ^^^tl-i^^;) o . . . q^^ (t) still has 
an attracting fixed point at 0. So the periodic point x for the induced flow corresponds to an 
orbit which tends towards the equilibrium point S as t — > oo for the original flow. (Because of 
the paramctrisation, orbits actually reach E in flnite time; during this time the orbit switches 
infinitely often between strategies.) 

On the other hand, if /? e (cr, 1), then there exists a. to G (0, 1) so that T (IV corresponds 
to (xoyto). It was shown in Sparrow et al [2008] that to is a hyperbolic attracting fixed point 
(and a repelling fixed point) of Qz- — » K"*", where to > is so that z = {z,to) G P. So if 
the periodic point T"(x) = x is sufficiently close to z then 1 1— > QT^-if^^) o • • • o g^it) still has an 
attracting fixed point t near to- It follows that to each periodic point x of the induced fiow is 
associated a periodic point {x,t) of the Poincare return map of the fiow (of the same period). 
In the same way one can prove that there are invariant sets which correspond to the second and 
third assertions of Theorem 3.1. □ 

5 Proof of Theorem 3.1 

5.1 A geometric description of the flow neeir the Jitter set 

Before doing a rather cumbersome explicit calculation in the next subsection (for the game under 
consideration), we first want to explain geometrically what the dynamics near Z* looks like. As 
before let Z' = Zj^^ x Z^^ be a codimension-two plane where both players are indifferent between 
two strategies. In this section we consider the situation that near part of this set where both 
players are indifferent, both players choose repeatedly the strategies in a period four pattern 
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Figure 5: The targets P^,Pf,Pl^,PP in the plane 5 arc drawn marked as •. The dashed hne corre- 
sponds to Z. A quadrangle Q in S* is also drawn and also another quadrangle Qz = pr~^(Q) fl which 
is in the plane So 3 zq. The cone of the quadrangle with apex T is invariant under the flow. In the figure 
we have taken the case that and d\ are equal to 1. 

{i,k),{i,l),{j,l),{j,k). Let S be the two-dimensional plane spanned by [P^,Pf-] x [P^,P;^] C 
x'Eb- The orbit segments aim to the four targets in S, but an orbit which starts in Z' remains 
in Z' and aims for the point T := S D Z' . We call this point the cone-target. The linear (affinc) 
spaces S and Z' axe of complementary dimensions and transverse. Let L be a line through the 
cone-target T contained in Z', and let W be the three dimensional space W = S + L. Take 
Zq € L, and assume that on some neighbourhood U C W of zo, the players only choose the above 
strategies. In other words, U\Z' consists of four components, Sst where s G {«, j} and t G {k, 1} 
such that players A and B aim for resp. Pf in Sst- First we prove that each orbit in U lies 
within a cone with apex T (the cone-target) induced over some quadrangles Q, see Figure 5. 

Let us define these quadrangles Q. To do this, it is convenient apply a translation to 5 C x 
Ss which puts T=Sr\Z' Bs the origin (0, 0) of 5 C and identifies S=[P{', P/] x [P^ , P^] 
with a rectangle in M x M so that {P^,Pl^), {P{^,PP), {Pf.PP) and {Pf,P^) correspond to 

for some (11,(1^, (1^,4 ^ with > and 44 > 0- 
(The signs of c?" , (i^ , (ii , (^2 depend on whether the orbits flow clockwise or anticlockwise along 
L.) Now let Q be the quadrangle with corners 

(d^,0),(0,|4),(-|d5,0),(0,-d?) (5.5) 

(or a multiple of it). Note that the four corners of this quadrangle lie on the coordinate axes 
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and the sides of this quadrangle are parallel with the vector pointing from O to (P/, PP) in the 
region Sgt corresponding to the projection of Sgt along the direction L. The quadrangle Q is 
shown in Figure 5. 

Next take a plane S^} <Z W through zq transversal to L and take the projection pr : So ^ S* 
along L. Furthermore, take z € Sq, take the multiple eQ of the quadrangle Q in 5 which contains 
pr(2;) e 5 and define a quadrangle = pr~^(eQ) fl Sq, see Figure 5. Each z € So sufficiently 
close to Z{) is contained in some quadrangle in this way. These quadrangles in 5*0 have the 
following two property: (1) their corners are contained in two lines in Sq which are orthogonal 
to each other and (2) the quadrangles are self-similar (they are all scalings of each other around 
the common 'centre point' Zq)- 

The reason these quadrangles are important is the following: Consider the cone (^(Qz) over 
Q2 with as apex the cone-target T. Since the 'vertical' sides of this cone are contained in planes 
through the cone-target T and the targets {Pf,P^), for each z in this side the vector pointing 
from z to {Pf,Pf) is contained in the side of the cone. It follows that orbits which start in 
C{Qz) remain in C{Qz) until such time as one or both of the players start to play a third strategy. 





Figure 6: Arbitrary quadrangles can be mapped to standard quadrangles {{x,y) e R^; |,x| + \y \ = r} by 
piecewise linear maps. An arc connecting 011c of the corners of Q to some point z e Q is also drawn 
(with dotted points). The angle of z is equal to the Z(7)/Z(Q) where I is the usual Euclidean length in 

So consider such a family Q of quadrangles. Let us define a natural metric in 5*0 associated 
to these quadrangles in the following way. The family Q of quadrangles can be mapped to the 
standard family of quadrangles {{x, y) <E M^; + \y\ = r} by a map L which restricted to each 
quadrant is linear, see Figure 6 and with L(zo) = 0. Thus we can define for each z G 5*0, 

\\A\so = \\Lz\\ 

where || • || is the sum-norm on M^: \i w = {x,y) then ||w|| = |a;| -|- \y\. So ||^;o||so = ^'^d the 
set {w e 5*0; ll^i'llso = ^} is exactly a quadrangle from the above family. By analogy to the usual 

polar coordinates, we can associate an angle to z €E 6*0 \ {^o} in the following way. Pick one of 
the corners q of the quadrangle Q = {w G Sq; \ \w\\g^ — r} where r = H-zHsoi 7 ^le the curve 

on this quadrangle which connects q to z (anti-clockwise). Then define cf){z) = 27r ^Z^'l where I 

stands for the usual Euclidean length in M^. Thus we have defined quadrilateral polar coordinates 
(r, <p) of z G So \ {zo} as follows: 

where Q = e ^o; ||w||so = ''I = Ikllso- (This is completely analogous to how the usual 

polar coordinates are defined.) 



16 



Proposition 5.1 (Poincarc transition map for two planes parallel to S.). Take two points zq, z'q £ 
L so that along the segment [zo,Zq] no other strategy becomes preferential (or indifferent) to the 
strategies i, j for A and k, I for B. Let So, S'q be two-dimensional planes in W through zq resp. z'q 
which are both transversal to L. For z G S'q consider the quadrangle in Sq constructed above 
and let C{Qz) &e the cone o/Q^ over the cone-target T. Let R{z) be the Poincare map from Sq 
to S'q. Then R{z) is well-defined for z close to zq and is contained in C{Qz) fl Sq. Moreover, 
consider polar coordinates in the plane S'q with the distance and angle taken from the point S'qDL. 
Then one can take a continuous map 

S'o\{-Zo} 

so that for each z G ^qXI^o} the value o/^'(z) modulo 2tt is equal to the quadrilateral polar angle 
of R{z) & Sq \ {z'q} (as defined above). Then ^'(z) is equal to 



27r- ^ .y +B{z) + Bq. 
a ■ c{z) ■ r 



(5.6) 



Here r := \\z\\sg, c{z) is equal to co(l + 0{r)) where cq = dist(2;o,T)/dist(zo,T') S (0)1) ^.''T'd 
where 0{r) is a function with 0{r)/r bounded as r ^ 0, B{z): Sq \ {zq} ^ M. is continuous 
function on Sq with B{z) ^0 as z ^ zq, and Bq gR and a > are constants. Here dist is the 
usual Euclidean norm on the line L. 

So the angle of R{z) increases very fast as r = ||2;||so tends to zero. 




l2 



Figure 7: The spiral motion in S and the half-lines k emanating from the centre O. The transition map 
from some quadrangle Q to coQ is also drawn. 



Proof. The only thing we need to prove is that (5.6) holds. To do this, note that the vector 
field is the product of a vector field in a direction along L and one in the direction parallel to S. 
If we ignore the direction along L, then we get a new two-dimensional vector field on S which 
corresponds to a two-dimensional game with spiral behaviour, see Figure 7. In other words, if 
we define pv: W ^ S to he the (linear) projection along L and let {z,t) i— > ^t{z) be the flow 
through z, then pr projects the orbits of this flow in W to orbits of a two-dimensional system 
in S. Moreover = pr(Q2) is a quadrangle in 5 as defined above, where z := pr(z) € S. 
Denote the flow of this two-dimensional system through z by {z,t) i— > ^t{z). Consider the piece 
[0, A]3 ^t{z) of the orbit through z e Sq until it hits Sq. Then [0, A] 3 t ^ ^t(^) is an arc 
with z gQz and R{z) G c(z) • Qj where c{z) > is equal to cq{1 -\- 0(dist(2;, zq))) where 

Co := dist(S'o fl L, O) /dist(S'o n L, O) 
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and 0{x) is a function so that 0{x)/x is bounded when a; ^ 0. This holds because the angle 
between the 'vertical sides' of C{Qz) and L tends to zero as z G S tends to zq. (In fact, if 5*0 
and S'q are both parallel to S then c{z) is exactly cq.) 

So it suffices to consider the projected flow ^t(z) on S. That is. let us consider the Poincarc 
transition map R which assigns to a point z € Qz C S the intersection of CqQ^ with the orbit 
t I— > ^t{z)- Next consider the quadrilateral polar coordinates in 5 (with the origin centered 
at O) and let t (t>t{z) be the angle of <&t(z) (where we choose t <i>{z) continuous). Then 
the angular change <Pa{z) — </'o(-S) is equal to the 27r times the integer number of times the 
[0, ^] 3 t ^*(5) winds around O plus some number in [0, 27r]. To compute this integer number 
of winding, consider the four half-line Z,j through the equilibrium Eg of the two-dimensional game 
in S where one of the players is indifferent and the other player always prefers one strategy, and 
denote these by Im, m = 1, ... ,4 (numbered so the flow meets these half- lines periodically in 
this order). Given .x G Im, let /m(x) be the flrst time the flow meets Im+i mod 4- Let us identify 
these lines with [0, 1] where corresponds to the equilibrium £^5. Since going from 1^ to Im+i is 
just the stereographic projection from 1^ to Im+i through lines of the target in the next region, 
fm{x) is of the form 

r 

r I— > with m = 1, 2, 3, 4 

1 + a^r 

where r stands for the distance to the origin measured in the metric |I ■ (if, instead, we take 
the Euclidean metric then we get these maps are of the form r ^ ' ^ where 9m is related to 
the shape of the quadrangle). Since the composition of two Moebius transformations of the form 
r ^ r/{l + Qir) and r ^ r/{\ + Q2r) is equal to a; r/(l + (^1 + Q2)r, we get that the Poincare 
first return map / to li is of the form 

T 

fir) = f4 f3° f2° fi (r) = — where a ^ ai + a2 + + ua,. 

1 + ar 

Hence /"(r) = for all n. So let n be maximal so that /"(r) > cr where c is equal to the 

number c{z) from above. Then n is the maximal integer so that r/(l -|- nar) > c - r, i.e., 

ar \ c J 

In fact, (j){R{z)) — (p{z) is constant for z gQ, because the relative length of an arc in Q (as a 
proportion of total perimeter length of Q) is preserved under one central projections, sec Figure 8 
and therefore also under a composition of such maps. So in particular if cq is equal to 1/(1 + nar) 
then the angle of R{z) and z are the same modulo 27r. If 1/(1 + (n + l)car) < cq < 1/(1 + near) 
then the result follows from a simple geometric consideration, see Figure 8. Thus we have proved 
Proposition 5.1. □ 

Note that after n = [1/r] iterates of the map / one has /"(r) = ~ Cir where Ci G (0, 1) 

is equal to 1/(1 + o). During this time, the orbit has spiraled n times around O with each spiral 

between T,. and Tc^,.. The length (and so the time-length) of the two-dimensional orbit is roughly 
n • C2r ~ C2. Hence in one unit of time, the fiow moves a point z a definite factor closer to O. 

5.2 An analytic computation of the flow neeir the Jitter set 

In this section we will make some precise calculations for the periodic orbit F on the Jitter set for 
the induced flow on 9E. More precisely, we will consider the flrst return map to some flrst return 
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Figure 8: Relative length (i.e. quadrilateral angle) is preserved by central projection. Moreover, let 
Q = {z; \ \z\\s = r}. Then the length of the arc 7c between the vertical axis and the intersection of the 
line with cQ is equal to (I — c)dr whereas the length of Q is equal to V^cr, where is a constant. So 
the angle of 7c is equal to (1 — c)0/{\/2c) (plus a constant). 




Figure 9: The simpliccs XIa and Eg. The closed curve F travels in both triangles clockwise along the 
three legs. T reverses at the points and f^. 

section at some point in F. To do this, consider the hne Vq in where player B is indifferent 
between strategies 2 and 3 (it goes from i?^3 to Q23, see Figure 9 for the location of these points). 
Similarly, let Vi in Eg be where player A is indifferent between strategies 2 and 3 (it goes from 
R23 to Q23) ^-^d let V2 be the line in E^ where player B is indifferent between strategies 1 and 
3 (it goes from R^^ to Q31). Moreover, let 9E^ be the side of dT,A containing and similarly 
define 9Eg as the side of 9Eb containing Rfj . 

Let i?o be the first entry map Vq x c?Eg ^T,^ ^ ^1 Etnd i?i be the first entry map 
dT,]^ X Vi ^ V2 X dT,^ corresponding to the induced fiows on 9E. By the symmetry of the 
system, the first return map to Vq x dT,^ is equal to the third iterate of Ri o Rq. (Provided we 
make sure we choose the axis consistently.) 

The first leg of the orbit F (the one which is contained in x Z^^ C E^ x Eb and which 
is the first piece of the two legs of the orbit F shown in Figure 9) corresponds to the first entry 
map Rq. During the transition which corresponds to Rq, player A only chooses between strategy 
1 and 3 and player B only chooses between strategy 2 and 1. Note that as soon as the orbit hits 
TiA X Vi and until it hits V2 x E^, player A will only choose between strategies 1 and 2 (while 
player B still only plays 1 and 2). So the first entry maps to these sections allow us to consider 
the pieces of the orbit where each players only switch between two strategies. 
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Let us describe how much further or closer an orbits near T can get to T while it orbits nearby. 
To do this, define the sum-distance (i.e. the metric dist) on 9S C S C between two points 
z = {zi,...,Z6), w= {zi,...,W6) by 



dist(2;, w) 



E 1-^ 

= 1,...,6 



This metric is well-suited to dealing with quadrilaterals. Because of the discussion in the pre- 
vious subsection, there exist quadrilaterals in x dT,^ which are mapped by i?i into another 
quadrilaterals in dT,^ x Vi (and similarly for R2). Let us compute these quadrilaterals (up to 
first order). It will be important to be consistent in the choice so let us write 

V„xdE]!=[Ri,Q^,]x[Pf,Pi], 



v,xdj:l' = [Ri,,Q^,]x[p,^,pn 



•■23] 



and number the half-lines clockwise starting with the positive horizontal axis, see Figure 10. 
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is equal to the third iterate of Ri o iJo provided 



Figure 10: The first return map to [R23, Q23] x [Pf 
we identify [Rii,Qii] x [Pi,Pf] with [-R^iQ^] x [Pf,Pi'\ in a consistent way. For this reason we 
order the half-lines in the coordinate axis in the return sections as shown. The origins of these axes are 
{E^,Ri,}, {Rt^^E") and {E^,R?2). 



Proposition 5.2. For each e > small, there exists a quadrilateral Qy^ C Vq x dT,^ with 
corners in the coordinate axes, and such that the sum-distance of comers (1),(2),(3),(4) (labeled 
as in Figure 10 on the left) to {E-^,Rf^) are equal, up to terms of order e^, to 

2 (2 + /3)e 2e 2 (2 + /3)e 2e 

31 + /3 + /32' (2-/J)(l + /3)' 3 /3(1 + /3 + ' 2-/?' ^ ' ' 

Similarly there exists a quadrilateral Qvi,ijo C dT}^ ^ ^1 '"^^^^ comers in the coordinate axes, 
and such that the sum-distance of these comers (1),(2),(3),(4) (labeled as in Figure 10 in the 
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middle) to {Ri2,E^) are equal, up to terms of order e^, to 

2(2 -/3)e 
(l + /J)(2 + /3)' 

2 (4-4/J + /32)e ,2(2-^) 



e 



(5.8) 

2(2 -/3)e 
/3(l + /3)(2 + /3)' 

T/ie ^rst entry map i?o maps Qy^j into Qy^ . 




Figure 11: The simplices Ha and Sb- The hne Vb C is drawn in the left triangle and the line 

Vi C on the right, and the first four line segments of the orbit (starting at time to = at the point 
*) which are computed in the proof of Proposition 5.2 are shown. 

So this proposition gives precise information on how much further or closer one gets to T 
during the transition from Vq x dT,^ to dT,]^ x Vi. Remember that we saw in the previous 
subsection that the angle of Rq{z) depends extremely sensitively on e and so it essentially suffices 
to compare the size of the terms in (5.7) to those in (5.7). 

What we need to do in the proof of Proposition 5.2 is to associate to Rq invariant cones as 
in the previous subsection, and the corresponding quadrangles in Vq x d'E^ and in dJ^]^ x Vi. 
After that, we will do the same for the first entry map Ri. 

Proof. Since we want to consider the induced flow on 9S, we take a starting point (p'^jP^) G 
Vq X when considering the map Rq. During this part of the orbit, the orbit jitters around 

this first leg of F. To describe this precisely, we explicitly compute the quadrilateral from the 
previous subsection. As we have seen in the previous subsection the orbit is contained in a cone 
with apex the 'cone-target' which in this case is equal to (Ta,Tb) = {R12, Qis)- To compute this 
cone, let us take as a special starting point in Vb x the point pa = (1/3, 1/3, 1/3), ps = 
(1 — /? — e, 0, 1 + e) /(2 — f3) (where ps = -R13 when e = 0) and compute the first four pieces where 
this orbit aims for {P^,P^), {P^,Pf), {P^,P^) and (fg^.^a^) under the original flow (i.e. the 
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first four times when the players hit an indifference plane under the original flow). Since the 
calculations are rather laborious and it is easy to make a mistake, we did this by using Maple 
(the worksheet can be requested from the authors - and also is available on the first author's 
webpage). For simplicity we take the parametrisation p'^(tfe + s) = p^{tk){i -— s) + sP^ and 
p^{tk + s) = - s) + sPf^ for all t G (ife, ife+i) provided A resp. B aim for Pf-, P^ during 

this time interval. The first hitting time is at ii := t\ := e/(l + e), and then 

, / 1 1 l + 3e \ , A-/3 „ 

PA(ti) = , , , . , PB(ti) = - — ^,0, ; 



3(l + e)'3(l + e)'3(l + e); ' "^"^ " \2 - (3' ' 2 - (3 

The next time the players hit an indifference plane is at £2 := ^1 + ^2 where t2 e/(/?+/3e+l + 2e) 
and then 

\3 (3 + l3e + l + 2€ ' 3/3 + /3e+l + 2e ' 3/3 + /3e + l + 2e 

. ^ / P^+p\-l-3e + pe p + l3e + l + e \ 

PbK^^I- (^(/3 + /3e+i + 2e)(-2 + /J) ' ' {(3 + (3e + l + 2e){-2 + P) J 

Then it hits at 4 := ti+t2 + h with U := e/{0^ + I3'^e + (3 + 2 (3 e + e) and then pA (4) andpB(4) 
are equal to 

/I /3 + 3e 1 J l/3(l + 3e) 

V3/3 + /3e + e ' 3/3 + ^e+e ' 3/3 + /3e + e, 

/ (/32 + /32e_ i_3e + /3e)/3 e (1 + e) /3 



\^(-2 + /3)(/32 + /j2e + /J + 2/3e+e) ' /32 + /JSg + /? + 2 /3 e + e ' (-2 + /?) (/3 + /Je + e)^ 
and again at £4 := 4 + f4 where t4 := e/{(3 + (3e + 2e) and then pa(4) and PB(t4) are 

/I ;i + 3( 1 ,i l/3 + 3Jf+3e' 



- 



V3/3 + /3e + 2e' 3/3 + /3e + 2e' 3 /3 + /?e+2e 

/ /3 (/?2 + /32e-l-3e + /3e) (/3 + 2)e /? (1 + e) \ 

\^(/3+l)(-2 + /J)(/3 + /3e + 2e) ' (/? + 1) (/? + /Je + 2 e) ' (-2 + /?) (/? + /3 e + 2 e) J " 

Next we compute the cone. As mentioned, the cone-targets are 



and we compute the intersection of the line through {Ta,Tb) and the points {pA{ii),PB{ii))-i 
i = 1, . . . , 4 with the three-dimensional section Vq x S^. This gives an intersection point at ti, 

1/3 



P^ + p-pe + l + e' ' (3^ + p-pe + l + e ' ' + p - p e + 1 + e 

+ + + /g^e - /3 - 1 - /3 e - e pe {P+2) 

{P^ + p-pe + l + e){p + l){-2 + p) ' {p2 + p-pe+l + e){P+l) ' 
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(/3^ + /3 + l)(l + e) \ 
(/32 + /3-/3e + l + e) (-2 + /3) J ' 

at t2 the intersection point is 

(1/3,1/3,1/3), 

/ 2/3e + /33 + /32 + 2/?2£-/3-l-3e e (/3 + 2) 1 + e V 
\ (/3+l)2(_2 + /3) ' + ' -2 + /3y' 

at fa the intersection point is 

fl/3 + ^^,3 (/3^ + /3+l)/3 /33 + /32 + ;3_3e 



/ /3 (/34 + 2/33g + /33 + 2/32e-2/3e-/3 + e-l) e (/g^ + /g^ + 1) 

{P+l){-2 + p){p^+P + P^ + Pe~e) ' (/32 + /3 + /33+/3e-e) (/3+1) ' 

(/?2 + /3+l)(l + e)/3 \ 
(/?2 + /3 + /33 + ;3e_e) (-2 + /3) ) 

while at t4 the intersection point we get is the original starting point (this is not surprising 
because the orbit lies on the cone through these points): 

(1/3,1/3,1/3) , f^,0, ^ 



2-/J' '2-/3 

These four points in T,a x together with the apex (Ta, Tb) determine a cone (for each e > 0). 

However, remember we want to compute the cone for the induced flow. This means that we 
have to take the intersection of the lines from [E^,E^) through these points with dH. Since 
e > is small, these points will be contained in Vq x 5S^. This gives at ti, 

/33 + 2/3" + 2/3+l + 3/3e /J^ + 2/32 + 2/3 + 3/32e + 1 + 6/3e 



/33 + 2/32 + 2/3+2/32e + l + e + 6/3e ' ' /33 + 2/32 + 2/3+ 2/32e + 1 + e + 6/3e 

/33 + 2/32 + 2/3 + 3/32e+l + 3e + 9/3e ^ 
' /33 + 2/32 + 2/3 + 2/32e+l + e + 6/3e 



at 12, 



-2 + /3 ' '2-/3 



(1/3,1/3,1/3), 

/33-/3-l + 3/32e-7e + 2/3e + /32 ^ 2/3 + 1 + 5e + /32 + 2/3e 



(/32 + 2/3+l + 3/3e + 6e)(-2 + /3) ' ' (/32 + 2/3 + 1 + 3/3e + 6 e) (-2 + /3) 



at is, 



, ' 2/33 + 2/32 + /3 + /34 + 4/32e+2e + 3/33e ' ^ 2/33 + 2/32 + /3 + /34 + 4/32e + 2e + 3/33e 
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(3 (2/3^ + 2/3+l + /33 + 3/3e + 3/3^£-3e) 
' 2/33 + 2/32 +/3 + /34 + 4^2e + 2e + 3/33e 

^ ^ 
2-/3 ' ' 2-/3 



and at £4 again the point we started with 



(1/3,1/3,1/3) , ( / , , ^ ' 



-2 + /3 ' '-2 + /3 

These four points determine a quadrangle Qy^ in Vq x dT}g. 

Next we find the intersection points of these cones with x Vi and then take the intersections 

with 9S of half-hnes from E in the direction of these points. Since the expressions are rather 
similar to the ones before, we will only give these final points in dT,B x V\. The intersection 
corresponding to ii is 



to i2 is 



(3e + /3 + l)(/3+2) ' ' (3e + /3 + l)(/3+2) 

_J_ P±l 
/3 + 2 ' ' /3 + 2 



^^ 3/3^6 -9/3£ + l + 9e + /33 + 3/33e + /3 + /3-^ 3/32e - 6/3e + 1 + 3 e + /33 + 3/33e + /3 + /3' 



2/32e-5/3e + l + 5e + /33 + 3/33e + /3 + /34 ' ' 2 f3'^e - 5 P e + I + 5e + + 3 P^e + P + P'^ 

l + /33 + /3 + /34 + 3e + 3/33e 



1/3 



2/32e-5/3e+l + 5e + /33 + 3/33e + /3 + /3^ 



to ts, 



2/3e + /32 + /3 + 2e 3/3^6 + 4/3e + /33 + 2/3^ + /3 - 2 e ^. 

(3e + /3 + l)/3(/3+2) ' (3e + /3 + 1) /3 (/3 + 2) J , IV^.V^^V-^J 



and to t4, 



^ ^ 
/3 + 2 ' ' P + 2 



6/3e+l-3e + /33 , /3^-/3+l + 3e 1 + /33 - 3 /3e + 3 e + 3 /32e 

' 2/3e+l + e + /32e + /33 ' / /32 - /3 + /3e + 1 + e ' ' 2/3e + 1 + e + /32e + /33 

These points form the quadrilaterals Qy^ in SS^ x Vi. 

To get Proposition 5.2 we differentiate the points forming the quadrilaterals Qvo.Aq and 
Qy^ _Ro ^ith respect to e. Since these points correspond to probability vectors, the sum of these 
derivatives is equal to zero. So we merely need to take the sum of the absolute values of these 
derivatives (or twice the positive terms). 

□ 
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Figure 12: The simplices and Es- The Une Vi C E^ is drawn in the right triangle and the line 
V2 C E^ on the left, and the first four line segments of the orbit (starting at time to = at the point 
*) which are computed in the proof of Proposition 5.2 are shown. Notice that ti, 12,13, Ia correspond 
to half-lines (2), (3), (4), (1) in the middle of Figure 10 and to (4), (1), (2), (3) on the right. This can 
be seen by taking the cones through the target points, intersect them respectively with E^ x V\ and 
V2 X E^, and then project from E on the boundary of E. 



Since the calculations from R\ are similar to those done in the Proposition 5.2 we shall not 
show them here. The maple worksheet in which these computations axe done can be obtained 

from the authors on request. 

Proposition 5.3. For each e > small, there exists a quadrilateral Qy^ C dJj^^ x Vi with 
corners in the coordinate axes, and such that the sum-distance of these corners (1),(2),(3),(4) 
(labeled as in Figure 10) to {Rf2,E^) are equal, up to terms of order e^, to 

2e 

2 + /3 

2 , , (0, V2) and for , . (1/2, 1), 

2(1 -/3)e 
2 + 13 ' 

Similarly there exists a quadrilateral Qy^ C 1^2 x dY}^ with corners in the coordinate axes, and 
such that the sum-distance of these comers (1),(2),(3),(4) (labeled as in the figure on the right 
in Figure 10) comers to {E^,Ri2) are equal, up to terms of order e^, to 

2 (4 - 3/32 - /33)e 2(2-/3-/32)e 



(5.9) 



31 + 3/3 + 3/32 + 2/33' (l + 2/3)(2-/3)/3' 

(5.10) 

2 (11 + 21/3 +15/32 + 7/33)6 2(2-/3-/32)e 
3(l + 3/3 + 3/32 + 2/33)(2 + /3)' (1 + 2/3)(l + /3)(2 - /3) ' 

The first entry map Ri maps Qy^ into Qy^ . 
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5.3 The dynamics of a Jitter map 

The dynamics near the periodic orbit T is very compHcated. As we have seen, the Poincare 
transition map to a section at a point of F is a composition of maps of the following form: 

xi-^ Aio R2^/\\y\\+Biy) o where y = Aq^{x). 

In this section we shall first study the iterations of one of these maps. In the next section we will 
then consider the composition of two suitable Jitter maps. 

Let dist be a metric on with the property that each half-line through the origin intersects 
dDr in a unique point (where Dr := {z e M^; \z\ := dist(z,0) = r}). Next take i?t: ^ 
to be the quadrilateral rotation, i.e. the unique map so that x M 9 {z,t) Rt{z) € is 
continuous, Rq = id, so that for each z G M^, dist(i?t(z), 0) = dist(z,0) and so that for each 
X e M^\{0} the angles of Rt{z) and z differ by t. If dist is the Euclidean metric then this coincides 
with the usual rotation, but in our setting it is convenient to take for dist the sum-metric on 
(defined by dist(z,z') \\z — z'\\ := \x — x'\ + \y — y'\ for z = {x,y) and z' = {x',y')) in which 
case t Rt{z) moves each point along a square dDr{0). 

Next define two homeomorphisms Ao,Ai: — > M.^ which preserve the axes and which map 
quadrilaterals containing with corners on the axes to quadrilaterals of the form dD^ ~ {(a;, y) G 

; + \y\ = r}. Assume that there exist A < 1 < /it so that for each t G (A, /x) there exists a 
smooth curve It through so that 

dist(Ao ^ o Ax(z),<S)=t- dist(2;,0) for each z^lt (5-11) 

and such that A^^ A\{lt) is transversal to dDj. for each r > 0. Consider 

F{z)= A^oRq(^-^oA^^{z) 

where 

e{w) = 27r/||w|| + B{w) , w = A^'^iz) and ||«;|| = dist(w,0) 

and w ^ B{w) G K is a continuous function which converges to zero as u; — > 0. We will refer to 

F as a 'jitter map'. 

Let us prove that such a Jitter map maps are 'chaotic'. 

Proposition 5.4 (A jitter map has many periodic orbits). Let F be as above and assume (5.11). 
Then the map F has periodic orbits of arbitrary period in each neighbourhood ofO. 

Proposition 5.5 (A jitter map contains a shift with infinitely many symbols). Let F be as above 
and assume (5.11). Then there exists No so that for each sequence fcj G N satisfying 

A < < At with ki > No 

there exist a sequence Si G (0, 1) and z ^ with \\F'^{z)\\ G ( fc.^i^^-. ; fTt) i >0. 

Proposition 5.6. Let F be as above and assume (5.11). Then there exists Nq so that for each 
sequence fcj G N there exists a sequence Si G (0, 1) so that 

A < —y— < o-^d ki > Nq 
ki 

there exists z ^0 with \\F^{z)\\ G , ) for all i>0. 
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The first proposition implies, for example, that there is a sequence of fixed points of F 
converging to 0. 

The second proposition implies that F contains a shift on infinitely many symbols. Indeed, 
define the annuli 

Anrik := e M^; _1_ < ||^|| < i}. 

The annuli with k even are all disjoint. Hence, taking ki E 21 in Proposition 5.5, we get the 
existence of a; € Anuka with F^{x) G Anuk^ for all i. If we would consider ki G {fcoi^o + 2} 
this would give a one-sided shift on two symbols, but the proposition guarantees the existence 
of orbits which jump several annuli further in or out (the number is determined by A and ji). It 
follows that F has positive topological entropy. In fact, the topological entropy is infinite because 
for each n, it contains a full one-sided shift of n symbols (which has entropy logn). 

Corollary 5.1 (A jitter map has sensitive dependence on initial conditions). Let F he as above 
and assume (5.11). Then F has sensitive dependence on initial conditions for all points in the 
set corresponding to the shift on infinitely many symbols. 



I 




Figure 13: On the left, the image of a curve I through is a spiral. A conveniently chosen curve I 
contains a sequence of fixed points, converging to 0. On the right, the sequence of annuli discussed below 
the statement of Proposition 5.5. Orbits can jump between annuli according to these allowed sequences 
(5.5). 



Proof of Proposition 5.4- Let us start by showing why F has a sequence of fixed points tending 
to 0. It will be convenient to consider 

F{y) —Ao R0(y){y) where A = A'^'^ o A^. 

By assumption there exists a curve I through so that ||A(z)|| = ||z|| for z €l. Let 1+ be one of 
the two components of I \ {0}, let I'j^ := A{1+) and let m+ := Ai{l+). We shall find a sequence 
of fixed points y of F on m+ \ {0}. Indeed, for each r > small, let a(r) be the angle between 
the vectors l[.0 and IrO where l'^. e l'_^_ and Ir S 1+ are the unique points so that ||^^|| = = ^• 
Then choose y G l'_^ so that 

0{y) = 27r /\\y\\+B{y)=a{\\y\\) mod 27r. (5.12) 

Since B{y) is bounded and continuous, there exists a sequence of such points on converging to 0. 
More precisely, there exists a so that for each sufficiently large G N there exists r € (^rpyqig, ^qig) 
so that y G l'^ with |y| = r satisfied (5.12). So assume that (5.12) holds. Then for any such 
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y e the point x := Aqu G m+ is a fixed point of F. Indeed, RQ^^y){y) = Ra{\y\){y) S I by the 
choice of y and a. So 

||F(y)|| = ||Aoii;,(,)(y)|| = \\Re(y){y)\\ = \\y\\ (5.13) 

and 

(5.14) 

Since y G and is a smooth curve which is transversal to the quadrangles D^, equations 
(5.13) and 5.14) implies that F{y) = A o Rg(^y){y) = y. Hence Aq o F{y) = ^0(2/) and so 
F{x) = Ao o F o ^(.t) = a; for a; = Ao{y) G m+. 

To explain the general case of periodic points of higher periods, let us show how to construct 
a periodic orbit of period three (the general case goes similarly) . Consider the surface 

P(3) = {(01,02,03) ; Oi € and 010203 = 1}. 

Choose 01,02,03 e (A,/x) on this surface P(3) and a disc neighbourhood U of this point in this 
surface. Associate to o = (oi, 02, 03) the curves i = 1, 2, 3 through such that ||^(2;)|| = Oi||2:|| 
for all z Gh and let be a component of U \ {0}. Let I'i + = A(?i,+). For each r > 0, let ai{r) 
be the angle between the Ol[^. and Ok+i^r where Z = I'^^CidDr and k+i^r = h+i,+ <^dDr (and 
where we take ^3+1 = h). Assume there exists r > and a G P{3) so that 

0{l[^^,) = ai(r) mod27r 

0{Ka2r) = a2(a2r) mod27r (5.15) 

^{l'3,a2asr) = Q!3(a2a3r) mod 27r. 

We claim that this implies that x := ^o(y) is a periodic point of F of period three where y = h^r- 
Indeed, because y = li^r = Z'l + H Dr the first equation from (5.15) implies that Rg(y-f{y) G I2.+ 
and therefore that ||-F(?/)|| = ||^oi?g|-,y-|(t/)|| = 02||y|| = 02r and F{y) — Ao Rg(^y-^{y) G Z2 +• This 
and the second equation from (5.15) implies that Rg(^p(^x)){F{y)) G and so \\F o F{y)\\ = 
\\A o R^^^^^.^^[F{y))\\ = a^\\\F{y)\\ — a203r. Finally, this and the third equation from (5.15) 
implies that Rp2^y^{r{y)) G /i,+ and so ||P'^(2/)|| = aia2a3||j/|| and F^{y) G Because 
010203 = 1 and both F^{y) and y are in l'^ _|_ we get therefore that F^i^y) — y. Hence F^[x) — 
AqO F^ o Aq^{x) = X. 

So we need to show that (5.15) has solutions. Define G: M+ x P(3) ^ M x K x K by 

G(r, 01,02,03) = (ai(r),a2(a2r-),a3(a203r)) 

where y G I'l ^ D Dr and are the angles defined above. This map is continuous and bounded. 
In fact, for each e > there exist 6 > and a neighbourhood U in P(3) as above so that 
G{{0,S) X U) is contained in e-ball in M^. Next define the map 

H:R+X P(3) 3 (r,a) ^ {9{l[,r),0{l'2,a2r),0{l'3,a2a,r)) ^ R X R X R. 

H can be written as Hi + H2 where 

Hi{r,a) = (27r/r,27r/(o2r),27r/(o203r)) and H2{r,a) = {B{l[^r),B{l'2,a2r),B{l'3,a2asr))- 
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Note that (5.15) is equivalent to G{r,a) = H{r,a), i.e. to G{r,a) — H2{r,a) = Hi{r,a) (modulo 
27r). The values of the left-side map (0,6) x U B {r,a) i-^ G{r,a) — H2{r,a) are contained in 
a 2e-ball, provided we choose 5 > so small that |-B(y)| < e for all y with \\y\\ < S. Note 
that Hi: (0,(5) x J7 ^ is invertible and has inverse H^^{yi^y2,yz) = (27r/t/i, 7/1/2/27 2/2/2/3) • 
Hence H^^ maps 27rA; + [0,27r]^, k = (/co,fci,^2) G into (0,6) x U provided \ki\ is large. 
So (G — H2) o H^^ maps 27rfc + [0, 27r]^ into some 2e ball. By Brouwer's fixed point theorem, 
(G — H2) o H^^ has a fixed point (modulo 27r) in 27rfc + Kq + [0, 27r]^ for some Kq G (where 
is arbitrary but with \ki\ large. Hence there exists a constant k so that for each fcg large, 
G{r,a) = H{r,a) (modulo 2n) has a solution (r, a) with r G ( ^ k +« ) ^'^^ a G U C P{3). 

□ 



Proof of Proposition 5.5. The proof of the second assertion also has a similar flavour, but to 
explain the proof more clearly we will assume that B{z) = and that the curve It as in (5.11) 
arc lines. Again write F = ^4^^ o F where A = o ^j^. Assume that /cq j ;•• • is a sequence as 
in the assumption of the proposition, and let Ko,Ki, - ■ ■ & (0, 1) be a sequence to be determined 
later on. Take Uq = {l/{ko + k + 1), 1/fco + kq) and inductively choose a sequence of intervals 
U\,U2, - ■ ■ so that Un is the set of all a„ e IR so that for all x G Uq and all G Ui, i = 1, . . . , n— 1, 

l/(oi...a„x) G {kn + Kn,kn + Kn + 1). 

Next let I'q = 1R+ and for z > 1, let k be the half-line in the positive quadrant with \ \Ax\ \ = ai\\x\\ 
on li. Define = Ali, i > 1 and for i = 0, 1, 2, . . . let aj be the angle between l'^ and Zj+i. Note 
that ai depends on and ttj+i. 

We want to show that for each n there exists a solution x G Ui and ai G Ui, i = 2,3, .. . of 
the system of equalities (analogous to (5.15)): 

2TT/X = ao(«i) mod 2Tr 
2'jT/(a2x) = ai(ai,a2) mod 27r 

(5.16) 

27r/(a„ 03022;) = Q!„_i(a„_i, a„) mod 27r. 

Let us show that this is enough. Take .x G Zg = with distance x to 0. Assume that we have 
F*(x) G Z- and ||F*(a;)|| — ai...aiX. Then the above equations give Rg^pi^^^^{F^{x)) G Zj+i, 

||Fi+i(a;)|| = ai+i\\F^{x)\\ = \ai---ai+ix\ for i = 0, 1, . . . ,n and F^+'^{x) G Z^+p Since x G Uq 
and a, G Ui, i = 1,2,... this proves by induction that 

irWI|c(^^,^). 

for each i = 0, 1, 2, . . . . 

To prove that for each n there exist x gUq and ai G Ui as in (5.16), let 

H{x, a2, . . . , a„) = (27r/x, 2?:/ (a2x), . . . , 27r/(a„a„_i • • • a2x)) 

and 

G(a;,a2, . . . ,a„) = (ao(ai), ai(ai, 02), a2(a2, 013), • • • , ct„_i(a„_i, a„)). 
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H is invertible with 

H~'^{yo,...,yn-i) = (27r/j/o,2/i/j/o,---,2/n-i/2/n-2)- 

Now take k = (ko, fci, . . . , fc„_i) with ki > Nq and with A < ki^i/ki < fi. For each e > there ex- 
ists A^o so that maps 2TTk+[—2TT, 47r]" into some e-neighbourhood of (fco, ki/ko, . . . , kn-i/kn-2) 
Then G maps this neighbourhood into a neighbourhood of some point in R". It follows that there 
exists K = (kq, . . . , G K" so that GoH~^ maps 27rfc + 27rK+ [0, 27r]" into itself, and therefore 

has a fixed point (j/o, • • • , Un-i) G 27rfc + 27r/5 + [0, 27r]". It follows G = H (modulo 27r) has as a 
solution (a;, 02, ... , a„) = H{yo, . . . , yn-i) of the required form. □ 

5.4 The dynamics of the composition of two jitter maps and the Proof 
of Theorem 3.1 

To prove Theorem 3.1 note that we have seen in Proposition 5.1 that the first entry maps Rq 
and i?i are of the form 

Aio i?2VI|!/||+B(y) ° ^0 where y = Aq\x) 

and 

Xh^ A30 i?27r/||y||+B'(y) ° i^) where y = A^^ix). 
The first return map to the section associated to Vq is the third iterate of i?i o Rq (provided we 
identify the target space of Ri appropriately with the domain space of Rq, as we have done in 
the previous subsection, see for example Figure 10). To show that this map has the required 
properties, we proceed as in the proof of Proposition 5.4 and write 

F{x) = Aq^ O A3 O -Rl/||y'||+B'(y') O ^2 ^ ° ^1 ° ^l/\\v\\+B{y)- 

Note that Ai is a piecewisc linear map (linear on each quadrant), and so we can describe these by 
four parameters (which determine the position of each corner of the quadrilaterals) . To compute 
the condition analogous to (5.11), for ^2 ^ o Ai we take the ratio of the i-th term in (5.8) to the 
i-ih term in (5.9): 

(2-/3) (2-/3) (2-/3) (2 - /3)/3 
(/3 + l)'(l-/32)'(/3+l)/3(l-/3)' (1-/32)- 

The largest one of these (the third one) is > 3.5 for all /3 e (0, 1) whereas the last one is < 1 for 
all (3 S (0, 1/2) (it is increasing) and the first one is < 1 for all /3(l/2, 1) (it is decreasing). So 
^Ai(2)|/|z| can vary between 1 and 3.5. 

To compute the condition analogous to (5.11), for A^^ o Aq we take the ratio of the i-th term 
in (5.7) to the i-th term in (5.10): 

(2 - /3 - /3^) (2-/3-/3^)(l + /3) /3(11 + 21/3 + 15/3^ + 7/3^) {2-f3- 0^) 
(1 + 2/3) ' (l + 2/3)/3 ' (2 + /3)2(l + 2/3) ' (1 + 2/3)(l + /3) ' 

The largest of these is either the 2nd or the 3rd, and the maximum of these two is > 1.2 for all 
/3 G (0, 1). The third one is increasing and the last one increasing, with the first one < 0.8 for 
l3 e (0,0.35) and the last one < 0.8 for /3 e (0.35,1). So [Ag ^^3(^)1/1^1 can vary between 0.8 
and 3.5. 

So the condition corresponding to (5.11) holds. It follows that F has a sequence of fixed 
points (and periodic orbits) as before, and that the properties as in Theorem 3.1 hold. 
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6 Proof of Theorem 1.4. 



Take matrices A and B with 

11^-^/311,11^-5^11 < e 

where || || stands for some matrix norm. 

That the Shapley and anti-Shapley orbit exists for A, B near Ap, Bj^, simply follows from the 
hyperbolicity of the first return map to a section transversal to these orbits. (The first return 
maps are projective transformations.) 

So let us discuss the persistence of the orbit F. Provided e > is small enough, the set 
Yja and are still divided up in three regions meeting in a Nash equilibrium as in Figure 1. 
(The angles between the lines will no longer be necessarily equal and the Nash equilibria will no 
longer be in the barycentre.) Now again consider the induced flow on the boundary. The set 
where one or two players are indifferent are still arranged as in Figures 2 and 3 (they change 
continuously with A and B). The part of 91] where both players are indifferent still consists of a 
closed curve F along which orbits spiral (along cones as in Figure 5), and the other part through 
which orbits cross transversally. The transition maps can be computed along this orbit as was 
done in Section 5.2, but in any case, the quadrangles computed in that section depend again 
continuously on A and B. So it follows that for fictitious play associated to A and B sufficiently 
close to A/SjBjs, one still has the existence of a sequence of periodic orbits for the flow induced 
on the boundary. 

Next we argue for the original system. The cone over the hexagonal F with apex the Nash 
equilibrium E is completely invariant and depends c;ontinuously on A, B. Moreover, this cone 
is two dimensional (but of course embedded in the four-dimensional space E). So now take a 
half-fine / in this cone through the apex, and consider the first return map to I. Because of the 
general form of the return maps, this first return map R: I I is a Moebius transformation, 
with a fixed point at E. As we showed in the appendix of Sparrow et al [2008], taking Afj,Bjj 
we have the following: for (3 G (0,cr) the fixed point E oi R: I ^ I is attracting, for /? = <t it is 
neutral, and for (3 G {a, 1) repelling and another fixed point appears which attracts all points in 
1\{E}, because the map is a Moebius map. (For (3 e (0, a), the map R: I ^ I also has a 'virtual' 
2nd fixed point, corresponding to the 'negative' part of the half-line I.) For A, B close to A^, Bp 
the corresponding first return maps are also near those of Ap, Bp. So when /3 ^ cr, and A, B is 
sufficiently close to Aff,Bp we have the same behaviour for F. 

Using Proposition 4.1 one can get the same conclusions for the other periodic orbits for the 
original flow. 

7 Conclusion 

For (3 G (—1,0] players always asymptotically become periodic. When f3 G (0,(j) the Shapley 
orbit is still attracting but not globally attracting: there is an abundance of orbits (many with 
periodic play) as described in Theorem 1.1 which tend to E. For (3 € {a, 1) we have chaos (in 
the sense that there exist subshifts of finite type). This chaos is caused by a periodic orbit F 
whose first return map is of what we call 'jitter type': orbits can move further away and closer 
to the periodic orbit F in a manner which is reminiscent to that of a random walk. Numerical 
simulations suggest that for /? e {a, r) this is precisely what happens for most starting points. 
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However, when /? G (r, 1) there exists an attracting anti-Shapley orbit, but again this orbit is not 
globally attracting. 

As shown in Theorem 1.4 the analysis we give does not depend on the symmetry of our 
matrices. The symmetric matrices Ap,Bp merely simplified our calculations, but using simple 
perturbation arguments the results also apply to neaxby maps. 

Given the relationship established in Gaunersdorfer & Hofbauer [1995] between fictitious play 
and replicator dynamics, it would be interesting to see whether chaos can also occur in the latter.^ 
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